Removing the threading.Lock locks and replacing them with RLock objects to avoid deadlocks. #3677
Conversation
…ts to avoid deadlocks.
petyaslavova
added
bug
There was a problem hiding this comment.
Pull Request Overview
This PR replaces all usages of threading.Lock with threading.RLock to prevent deadlocks, updating both tests and core locking interfaces. It’s a breaking change that adjusts constructor signatures and type hints to expect reentrant locks.
- Swapped
LockforRLockin multiple test files to align with new behavior. - Updated constructor parameters and type annotations in
redis/event.py,redis/connection.py, andredis/client.pyto useRLock. - Removed legacy non-reentrant lock branches and related TODOs in
redis/connection.py.
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_credentials.py | Changed mock pools’ _lock initialization from Lock to RLock. |
| tests/test_connection.py | Updated CacheProxyConnection instantiation to pass an RLock. |
| tests/test_cluster_transaction.py | Replaced mock_pool._lock = Lock() with RLock(). |
| redis/event.py | Refactored connection_lock parameter/type to RLock. |
| redis/connection.py | Adjusted pool_lock type to RLock; removed non-reentrant fallback code. |
| redis/client.py | Swapped client locks to RLock and removed outdated TODOs. |
Comments suppressed due to low confidence (3)
tests/test_credentials.py:326
- [nitpick] Add a unit test that acquires the same
RLocktwice in a row to verify reentrant behavior and ensure no deadlock occurs under nested acquisitions.
mock_pool._lock = threading.RLock()
redis/connection.py:813
- Update the constructor docstring for this class to document the change from
LocktoRLockin thepool_lockparameter, since this is now part of the public API and a breaking change.
pool_lock: threading.RLock,
tests/test_connection.py:496
- [nitpick] Consider extracting the repeated instantiation of
CacheProxyConnectionwith anRLockinto a pytest fixture to reduce duplication and improve clarity across tests.
proxy_connection = CacheProxyConnection(mock_connection, mock_cache, threading.RLock())
|
This PR fixes issue #3640 |
|
@petyaslavova - appreciate this is a breaking change but given the deadlocks introduced has broken production code for us (and, based on feedback on various OS repos, many others), do you have an estimate of when a v7.0.0 will be released? To avoid the deadlocks we had to downgrade from 6.x to 5.2.1 and I guess everyone on >5.2.1,<7.0.0 may potentially also have very hard to debug issues going forward (it took us forever to triangulate the issue to deadlocks in redis)? |
…ts to avoid deadlocks. (redis#3677)
…ts to avoid deadlocks. (redis#3677)
|
Thanks @petyaslavova @vladvildanov for the fix! I noticed it’s already included in 6.4.0, but it wasn’t mentioned in the changelog. Could you add it there so other users are aware as well? |
Pull Request check-list
Please make sure to review and check all of these items:
NOTE: these things are not required to open a PR and can be done
afterwards / while the PR is open.
Description of change
Usage of
threading.Lockhas introduced deadlocks in the client.With this change, I'm removing all places that can cause such issues.
The change is breaking, because it changes the interfaces of several objects and cannot be released before the next major release, which will be 7.0.0